1. Field of the Invention
The present invention relates generally to pattern recognition apparatus and pattern learning apparatus and, more particularly, to a pattern recognition apparatus and a pattern learning apparatus employing a neural net including a plurality of excitatory element-inhibitory element pairs. The invention has a particular applicability to a character recognition apparatus.
2. Description of the Background Art
Conventionally, a pattern recognition technology for recognizing image patterns, character or symbol patterns and speech patterns has been known, and recognition apparatus for recognizing those patterns have been developed. A conventional pattern recognition apparatus compares an applied input pattern with a dictionary pattern stored in advance and then detects the distance (for example, "Euclidean distance") or the similarity between the input pattern and the dictionary pattern, thereby determining classification of the applied input pattern. In general, however, characters, speech and the like which are used in practice by human beings are varied even in one classification. Pattern recognition technology in which recognition can be made in accordance with a dictionary pattern without being affected by the variations of characters and speech has not yet been developed.
It is reported that there occurs a significantly dynamic chaotic oscillation phenomenon represented such as by electroencephalogram (or EEG) in brains of creatures, especially of higher-level creatures such as human beings. It seems that a different processing from a recognition processing employing a conventional dictionary pattern is carried out in the brains of higher-level creatures.
In general, it is reported that when recognizing the same kinds of objects or concepts which are individually different and varied though, such higher-level creatures as human beings detect an invariable amount or invariance of the objects or the concepts which is not affected by such variation, and also detect continuity of the variation, thereby recognizing the objects or the concepts. In cognitive psychology, for example, it is reported that a continuous internal operation called "mental rotation" is present in the brains.
Further, it is reported that there occurs oscillation in a neural net including a plurality of mutually coupled excitatory element-inhibitory element pairs as a recognition model. This is described in the article by B. Baird entitled "Nonlinear Dynamics of Pattern Formation and Pattern Recognition in the Rabbit Olfactory Bulb," Physica, Vol. 22D, pp. 150-175, 1986.
In addition, a discovery that a learning of a continuously transformed pattern such as a time-series pattern is available in principle which continuously varies on time base is described in the article by M. Sato entitled "A Learning Algorithm to Teach Spatiotemporal Patterns to Recurrent Neural Networks," Biological Cybernetics, pp. 259-263, 1990. Moreover, another discovery that a lower-dimensional chaotic orbit called "Lorenz Attractor" is also obtained by employing the learning rules of the recurrent networks by M. Sato is also reported in the article entitled "A Learning of Approximation and Chaos of Nonlinear Dynamics by Recurrent Networks," IEICE Technical Report NC90-81, pp. 77-82, 1991. However, those articles have not yet reported that a dynamic orbit is obtained by employing a large number of practical patterns such as images and speech.
For recognizing an input pattern having a great variation such as characters or letters written by human beings, for example, no accurate classification is often made in a recognition method based on a detected correlation distance or similarity. However, it is considered that human beings generally recognize various transformation patterns by a continuous internal operation. Problems of the conventional method based on detection of correlation distances may be solved by employing such a continuous internal operation.
It is pointed out, however, that for realizing the continuous transformation by an oscillation of the neural net, it is difficult to analytically obtain oscillation conditions in the conventional neural net including a plurality of mutually coupled excitatory element-inhibitory element pairs. In addition, since all elements are completely coupled to one another via a number of multipliers in the neural net, a vast amount of calculation cannot be avoided. When a continuous transformation pattern that varies continuously is learned by a neural net, the calculation amount of a temporal inverse processing corresponding to a back propagation of a feedforward net, i.e., a processing that advances backward on time base becomes enormous, resulting in a need for a longer learning time.